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DETAILED ACTION 

1. The response received on March 19, 2004 has been placed in the file and was considered 
by the examiner. An action on the merits follows. 

Response to Arguments 

2. Applicant's arguments filed March 19, 2004 have been fully considered. A response to _ 
these arguments is provided below. 

Prior Art Rejections 

35 U.S.C. 102(b) rejections 

Summary of Argument: The applicant argues that the Office Action has incorrectly 
characterized an alleged capability of Wren et al. (the article titled "Pfinder: Real-Time Tracking 
of the Human Body") to automatically learn a color-based object model from a combination of 
generated state estimates and image observations (see applicant's remarks: page 3, paragraph 3). 
The applicant further argues that section 3 of Wren et al. states the case of tracking a person 
where finder has already found and built representations of the person and the scene, and 
therefore, can not describe learning such models (see applicant's remarks: page 4, paragraph 1). 
The applicant also argues that in section 4 of Wren et al, the actual creation of the "person 
model" is explained by describing a system which begins with some known or predefined 
contours corresponding to known parts of the human and then iteratively updated (see 
applicant's remarks: page 6, paragraph 1). 

Examiner's response: The examiner disagrees. Although it is correct that the system 
disclosed by Wren et al. begins with some known or predefined contours, the claim language 
does not exclude such information from being used since the claims use open-ended claim 
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language. In section 4.2 of Wren et al., it is described that the Pfinder system builds up a blob 
model of the person. Therefore, the limitation, "automatically learning the color-based object 
model using the state estimates and the observations" is met by the reference as will be further 
described in the rejection below. 
35 U.S.C. 103(a) rejections 

Summary of Argument: With regard to the rejections of claims 8, 10, 13, 14, 17, 18, and 
20-22, the applicant argues that the Wren-Birchfield combination still fails completely to 
disclose the applicant's claimed element of automatically learning color-based object models 
(see applicant's remarks: page 10, paragraph 3). 

With regard to claims 9, 36, and 37, the applicant argues that the Wren-Birchfield-Koller 
combination still fails completely to disclose the applicant's claimed element of automatically 
learning color-based object models (see applicant's remarks: page 1 1, paragraph 3). 

Examiner's response: The examiner disagrees. As described above, Wren et al. discloses 
automatically learning the color-based object model using the state estimates and the 
observations. Therefore, each of the combinations discloses this feature. 

Claim Rejections - 35 USC §102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

4. Claims 1-7, 15, 16, 19, 28, 29, 33-35 rejected under 35 U.S.C. 102(b) as being anticipated 
by Wren et al. (the article titled "Pfinder: Real-time Tracking of the Human Body"). 
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As applied to claim 1, which is representative of claims 15 and 16, Wren et al. discloses a 
system for tracking at least one object in at least one sequential image, comprising: a general 
purpose computing device (see page 6, section 8: The reference describes the use of a 200 MHz 
R4400 processor Indy with Vino Video (i.e. a general purpose computing device).); and a 
computer program comprising program modules executable by the computing device, wherein 
the computing device is directed by the program modules of the computer program to: (a) 
generate a state estimate defining probabilistic configurations of each object for each sequential 
image (see page 2, section 3.1: The reference describes determining a the blob spatial 
characteristics of an object in an image. This corresponds to generating a state estimate defining 
probabilistic configurations (see equation (1)).); (b) generate observations of pixel color for each 
sequential image (see page 4, section 4.2: The reference describes measuring the Mahalanobis 
distance of new pixel values in color space from the appropriate location in the scene (i.e. 
generating observations of pixel color).); (c) automatically learn a color-based object model 
using the state estimate and the observations (see page 4, section 4.2: The reference describes 
that the Mahalanobis distance (i.e. color observations) and blob size (i.e. state estimates) are used 
by the Pfinder system to build a model of the person. This the Pfinder system performs these 
steps independently of operator intervention, the system automatically learns the model.); and (d) 
automatically track each object using the learned color-based model with a color-based tracking 
function (see page 3, section 3.3: The reference describes the use of an analysis loop to track an 
object in an image based on the person model (i.e. color-based model) using tracking functions 
described by the equations (4), (5), and (6).). 
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As applied to claim 2, Wren et al. discloses that generating the state estimate comprises 
determining the probabilistic configurations of each object using an initial image processing 
program module (see section 8: The reference describes the use of a 200 MHz R4400 processor 
Indy with Vino Video. This computer has all of the program modules including an initial image- 
processing unit). 

As applied to claim 3, Wren et al. discloses that the initial image processing program 
module employs a tracking system comprising a tracking function in combination with an object 
model for probabilistically detecting object configuration information (see page 3, section 3.3: 
The reference describes that the tracking system uses a likelihood function (equation (5)) (i.e. a 
tracking function) in combination with the blob models (i.e. object models) for detecting object 
configuration information.). 

As applied to claim 4, Wren et al. discloses that the initial image processing program 
module employs a contour-based tracking function in combination with a contour-based object 
model for probabilistically detecting object configuration information (see page 3, section 3.3: 
The reference describes the use of connectivity constraints (i.e. contour-based tracking 
function).). 

As applied to claim 5, which is representative of claim 19, Wren et al. discloses that 
generating the observations of pixel color comprises collecting pixel color information over the 
entirety of each image (see page 3, section 3.2: The reference describes that a model of the scene 
is generated that is partially based on the color distribution of each pixel in the image.). 

As applied to claim 6, Wren et al. discloses that generating the observations of pixel color 
comprises collecting pixel color information over specific portions of each image (see page 2, 
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section 3.1: The reference describes that each blob (i.e. specific portions of each image) has a 
color component.). 

As applied to claim 7, Wren et al. discloses generating the observations of pixel color 
employs the state estimate to identify specific relevant regions of each image over which pixel 
color information will be collected (see page 2, section 3.1: The reference describes that each 
blob has a spatial characteristic (i.e. state estimate) which identifies a relevant region in the 
image. A color component is determined for each spatial characteristic). 

As applied to claim 28, Wren et al. discloses that generating the state estimate comprises 
processing each image with an initial object model and an initial tracking function (see page 2, 
section 3: The reference describes a "steady-state" case in which the system has built 
representations of a person (i.e. an initial object model and tracking function) and uses them to 
initialize the system (i.e. generate the state estimate).). 

As applied to claim 29, Wren et al. discloses that the initial object model is iteratively 
replaced with the learned color-based object model and the initial tracking function is replaced 
with a color-based tracking function to improve the accuracy of the learned color-based object 
model (see page 3, section 3.3: The reference describes that the person models are continuously 
updated (i.e. iteratively replaced) to improve the accuracy of the learned color-based object 
model.). 

As applied to claim 33, Wren et al. discloses a process for gathering the sequential 
images (see page 6, section 8: The reference describes the use of a single CCD, color camera for 
gathering the sequential images.). 
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As applied to claims 34 and 35, which merely disclose a computer-readable memory 
comprising a computer program comprising program modules that perform the processes of 
claims 15 and 28, Wren et al. discloses such a computer-readable memory since a computer 
performs all of the processing in Wren et al. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 8, 10, 13, 14, 17, 18, and 20-22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Wren et al. (the article titled "Pfinder: Real-Time Tracking 
of the Human Body") and Birchfield (the article titled "Elliptical Head Tracking Using Intensity 
Gradients and Color Histograms"). The arguments as to the relevance of Wren et al. in the 
rejection of claims 1 and 15 above are incorporated herein. 

Claims 8, 10, 13, 14, 17, 18, and 20-22 call for the use of histogram processing when 
generating the color observations. Wren et al. describes that color observations are taken into 
account when determining the object model; however, the reference does not disclose the use of 
histogram processing. Birchfield, in the same field of endeavor of object tracking and the same 
problem solving area of modeling objects using color, describes the use of such histogram 
processing as detailed below with reference to each of the claims. 

As applied to claim 8, which is representative of claim 21, Birchfield discloses that 
generating the observations of pixel color comprises automatically generating a first probability 
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distribution function modeled using a first histogram to represent a range of observed pixel 
colors (see section 4: The reference describes the use of a histogram for generating color 
observations of the object. A histogram, as described in the claim, is a probability distribution 
function.). 

As applied to claim 10, which is representative of claim 22, Birchfield discloses program 
module for automatically learning the color-based object model automatically computes a second 
probability distribution function modeled using a second histogram to represent a background for 
each image (see section 4: The reference describes forming an image histogram, which is a 
histogram of the entire image minus the model (i.e. a background image). Also, as described 
above, a histogram is a probability distribution function.). 

As applied to claim 13, Birchfield discloses that automatically learning the color-based 
object model comprises performing a bin-by-bin comparison between the first histogram and the 
second histogram (see section 4: The equation for determining O c performs a bin-by-bin 
comparison between the model histogram (i.e. the first histogram) and the image histogram (i.e. 
the second histogram).). 

As applied to claim 14, Birchfield discloses that bins in the first histogram having values 
exceeding corresponding bins in the second histogram correspond to those color ranges 
representing the learned color-based object model (see section 4: The reference describes that the 
equation for determining <X> c is used, in part, to represent the color-based object model.). 

As applied to claim 17, Birchfield discloses that a confidence measure is associated with 
the observations of pixel color (see section 4: The reference describes that an offline, as well as 
an online, histogram is obtained. The offline histogram is used as a confidence measure that the 



Application/Control Number: 09/592,750 Page 9 

Art Unit: 2621 

online histogram (i.e. observations of pixel color) is actually a histogram representation of the 
head.)- 

As applied to claim 18, Birchfield discloses that the observations of pixel color are 
weighted in proportion to the confidence measure (see section 4: The equation O c is used to 
weight the online histogram (i.e. observations of pixel color) by the values of the offline 
histogram (i.e. the confidence measure).). 

As applied to claim 20, Birchfield discloses that the observations of pixel color are 
collected over specific portions of each image wherein the state estimate has a probability greater 
than a minimum threshold level (see section 4: The reference describes determining a model 
histogram by counting the pixels inside the ellipse (i.e. collecting pixel color information over 
specific portions of each image). The ellipse corresponds to the state estimate. Therefore, an 
ellipse will only be formed if the state estimate exists, so the state estimate has to have a 
probability greater than zero (i.e. a minimum threshold level).). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Wren et al. by adding the use of histogram processing as 
described in Birchfield because the use of such processing steps allows for the color measure to 
be "more satisfied with a region containing both facial and hair color than a region containing all 
facial color" (see Birchfield: section 4). Therefore, the tracking system would provide more 
accurate results since color observations are made for more then just the facial color. 
7. Claims 9, 36, and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Wren et al. (the article titled "Pfinder: Real-Time Tracking of the Human Body") 
and Birchfield (the article titled "Elliptical Head Tracking Using Intensity Gradients and Color 
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Histograms") and further in combination with Roller et al. (the article titled "Using learning for 
approximation in stochastic processes"). The arguments as to the relevance of Wren et al. in the 
rejection of claims 1 and 34 above are incorporated herein. 

Claims 36 and 37 call for the representation of pixel color information and background 
image information to be determined using a probability distribution function. This element is 
absent from Wren et al. but is described in Birchfield (see section 4: The reference describes 
using a histogram (i.e. a probability distribution function) to represent the model (i.e. pixel color 
information) and the image (i.e. background information). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Wren et al. by adding the use of probability distribution functions 
to represent color as taught in Birchfield because the use of such processing steps allows for the 
color measure to be "more satisfied with a region containing both facial and hair color than a 
region containing all facial color" (see Birchfield: section 4). Therefore, the tracking system 
would provide more accurate results since color observations are made for more then just the 
facial color. 

Claim 9, as well as claims 36 and 37, calls for the use of a Dirichlet function. The 
combination of Wren et al. and Birchfield does not teach the use of a Dirichlet function; 
however, Roller et al., in the same field of endeavor of image processing, and the same problem 
solving area of object tracking, describes the use of such a function (see column 10: The 
reference describes the use of a Dirichlet prior for use with a multinomial distribution.). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify the combination of Wren et al. and Birchfield by adding the use of 
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a Dirichlet function as taught by Roller et al. because the use of a Dirichlet function "serves to 
'spread out' some of the probability mass over unobserved states, increasing the amount of 
exploration done for unfamiliar regions of the space" (see Roller et al.: column 1 1). 

Allowable Subject Matter 

8. Claims 11, 12, 23-27, 30-32, and 38-43 are objected to as being dependent upon a 
rejected base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

Conclusion 

9. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J. Miller whose telephone number is (703) 306-4142. The 
examiner can normally be reached on M-F 8:00-4:30. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo H. Boudreau can be reached on (703) 305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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Examiner 
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